AMENDMENTS TO THE CLAIMS 

Claims 1-8 (Canceled) 

9. (Currently Amended) An ingredient cooking-operation recognition system 
comprising: 

a sensing means for acquiring observation data which includes at least either of an 
image around the hands of a person who performs a cooking operation and an 
environment sound that is produced by the cooking operation; 

a feature-quantity template in which various cooking operations that are 
predetermined for various ingredients are stored in a tabl e form, together with a template 
certainty factor that is a certainty factor which is predetermined for each cooking 
operation of the ingredients; 

a cooking-flow creating means for, based on the observation data which is 
acquired by the sensing means, calculating an observation certainty factor which 
indicates the certainty factor of at least either of an ingredient that is cooked by the 
person and the cooking operation of the ingredient, and based on this observation 
certainty factor, creating a cooking flow for a dish which is prepared by the person; 

a primary certainty-factor calculating means for, based on the template certainty 
factor and the observation certainty factor, calculating a primary certainty factor which is 
a certainty factor for each of objects that make up the cooking flow which is created by 
the cooking-flow creating means; and 

a cooking-operation recognizing means for calculating a final c e rtainty factor 
recognizing the ingredient and the cooking operation that correspond to the observation 
data which is acquired by the sensing means, based on the primary certainty factor which 
is calculated by the primary certainty-factor calculating means , updating th e f e atur e- 
quantity t e mplat e by consid e ring this final c e rtainty factor as a n e w templat e certainty 
factor, and bas e d on this f e atur e quantity t e mplat e , r e cognizing th e ingr e di e nt and th e 
cooking op e ration that correspond to th e obs e rvation data which is acquir e d by the 
s e nsing m e ans . 
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10. (Currently Amended) The ingredient cooking-operation recognition system 
according to claim 9, further comprising: 

a cooking-flow database which stores cooking flows which are created in advance 
for various dishes; and 

a relevance-factor calculating means for, using flew — matching mapping , 
calculating a relevance factor that indicates which of the cooking flows that are stored in 
the cooking-flow database is closest to the cooking flow which is created by the cooking- 
flow creating means, 

wherein the cooking-operation recognizing means r e cognizing an ingr e dient and a 
cooking op e ration x alculates a final certainty factor based on the relevance factor which 
is calculated by the relevance-factor calculating means and the primary certainty factor 
updates the feature-quantity template by considering the final certainty factor as a new 
template certainty factor, and based on the feature-quantity template, recognizes an 
ingredient and a cooking operation . 

11. (Previously Presented) The ingredient cooking-operation recognition system 
according to claim 9, further comprising: 

a table storing means for storing at least any one of: 

an ingredient table which is formed by an ingredient 
record that has a field where data that indicates an ingredient is 
stored and a field where various kinds of data on the ingredient are 
stored together with a template certainty factor that is 
predetermined for the data; 

a cooked-food table which is formed by a cooked-food 
record that has a field where data that indicates a cooking operation 
is stored and a field where various kinds of data on a change in an 
ingredient by the cooking operation are stored together with a 
template certainty factor that is predetermined for the data; and 

a cooking-operation table which is formed by a cooking- 
operation record that has a field where data that indicates a cooking 
operation is stored and a field where various kinds of data on an 
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environment sound that is produced by the cooking operation are 
stored together with a template certainty factor that is 
predetermined for the data; 
a cooking-recipe database which stores data that indicates an ingredient which is 

used for various cooking recipes and data that indicates a cooking operation for the 

ingredient; 

an extracting means for extracting all kinds of ingredients and all kinds of 
cooking operations which are stored in the cooking-recipe database; and 

a feature-quantity template creating means for creating, as a feature-quantity 
template, a table which includes, as items, each ingredient that is extracted by the 
extracting means and each cooking operation that is extracted by the extracting means, 

the feature-quantity template creating means writing, in an optional field of the 
fields which make up the feature-quantity template, as the template certainty factor, the 
greatest certainty factor of the template certainty factors which are stored in each field of 
at least any record of the ingredient record of the ingredient table that stores the data 
which indicates the ingredient that corresponds to the optional field, the cooked-food 
record of the cooked-food table that stores the data which indicates the cooking operation 
that corresponds to the optional field and the cooking-operation record of the cooking- 
operation table that stores the data which indicates the cooking operation that corresponds 
to the optional field. 

12. (Previously Presented) The ingredient cooking-operation recognition system 
according to claim 9, characterized in that: 

the sensing means has a thermal camera and an optical camera; and 

an image which is acquired by the sensing means includes a thermal image which 

is photographed by the thermal camera and an optical image which is photographed by 

the optical camera. 

13. (Previously Presented) The ingredient cooking-operation recognition system 
according to claim 12, characterized in that the cooking-flow creating means: removes a 
background region from the optical image by executing a background difference 



7 



processing on the optical image; removes, from the thermal image, as a background 
region, a region where the temperature is equal to, or higher than, a predetermined value; 
calculates an ingredient region which indicates an ingredient by multiplying the thermal 
image whose background region is removed and the optical image whose background 
region is removed; and calculates an observation certainty factor based on this ingredient 
region. 

14. (Previously Presented) The ingredient cooking-operation recognition system 
according to claim 9, characterized in that the cooking-flow creating means obtains a 
histogram of hue and a histogram of saturation from an image which is acquired by the 
sensing means, and estimates an ingredient by obtaining a correlation between these 
histograms and a color-feature template which is predetermined for each ingredient. 

15. (Previously Presented) The ingredient cooking-operation recognition system 
according to claim 10, characterized in that: 

the relevance-factor calculating means estimates a cooking recipe based on a 
relevance factor; and 

a guidance means is further provided for giving, to the person, guidance on the 
cooking recipe which is estimated by the relevance-factor calculating means. 

16. (Currently Amended) A computer-readable recording medium which is recorded 
with an ingredient cooking-operation recognition program, characterized by allowing a 
computer to function as: 

a sensing means for acquiring observation data which includes at least either of an 
image around the hands of a person who performs a cooking operation and an 
environment sound that is produced by the cooking operation; 

a feature-quantity template in which various cooking operations that are 
predetermined for various ingredients are stored in a tabl e form, together with a template 
certainty factor that is a certainty factor which is predetermined for each cooking 
operation of the ingredients; 
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a cooking-flow creating means for, based on the observation data which is 
acquired by the sensing means, calculating an observation certainty factor which 
indicates the certainty factor of at least either of an ingredient that is cooked by the 
person and the cooking operation of the ingredient, and based on this observation 
certainty factor, creating a cooking flow for a dish which is prepared by the person; 

a primary certainty-factor calculating means for, based on the template certainty 
factor and the observation certainty factor, calculating a primary certainty factor which is 
a certainty factor for each of objects that make up the cooking flow which is created by 
the cooking-flow creating means; and 

a cooking-operation recognizing means for calculating a final c e rtainty factor 
recognizing the ingredient and the cooking operation that correspond to the observation 
data which is acquired by the sending means, based on the primary certainty factor which 
is calculated by the primary certainty-factor calculating means , updating th e f e atur e 
quantity t e mplat e by consid e ring this final certainty factor as a n e w t e mplat e c e rtainty 
factor, and based on this f e atur e quantity templat e , r e cognizing th e ingr e di e nt and th e 
cooking op e ration that correspond to the obs e rvation data which is acquir e d by th e 
sensing m e ans . 
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